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(54) Error concealment for block-based image encoder 



(57) A method, for use in an image signal decoding 
system, for concealing errors in an image signal trans- 
mitted in a biock-by-k)lock compressed form, the decod- 
ing system including a decoding circuit for decoding the 
transmitted image signal into a decoded image signal 
on a block-by-block basis and detecting an erroneous 
block which contains an error, the erroneous tHock hav- 
ing a predetermined number of sun-ounding blocks with- 
out containing an error therein, each of the surrounding 
blocks having a multiplicity of pixels therein, the method 
comprises the st^ of: classifying each of the sur- 
rounding blocks into a monotonous, an edge or a tex- 
ture dass, the monotonous dass representing a block 
without containing an edge therein, the edge dass rep- 



resenting a t)lock containing one edge therein, and the 
texture dass representing a block containing more than 
one edge therein; classifying the erroneous block as the 
edge class, if only one pair of the surrounding blocks 
positioned opposite each other with respect to the erro- 
neous block is of the edge class and has an identical 
edge direction transversing the erroneous block; and, 
othenvise, as either the monotonous or the texture dass 
depending on the number of the surrounding k>locks 
having either the monotonous or the texture dass; and 
rectifying the erroneous block based on the dassifica- 
tion thereof. 
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Description 

The present invention relates to an apparatus for decoding an image signal, and, more particularly, to an apparatus 
for concerning or rectifying diannel errors present in a decoded image signal which has been transmitted from a block- 
5 based image signal encoder 

A typical block-based image signal encoder conforming to the MPEG(Moving Pictures Expert Group) standard per- 
forms a source coding which comprises a cascade of transform operation, quantization, and entropy coding. Specifi- 
cally, an image is segmented into a plurality of non-overlapping blocks; then each block is transformed via DCT(Discrete 
Cosine Transform) to remove spatial redundancies therein; and, subsequently, the DCT coefficients are quantized and 
10 entropy coded using variable-length codewrods. Thereafter, the encoded image siganl is transmitted to a decoder via a 
transmission channel 

Since, however, channel errors are apt to occur during the transmission process, in order to reduce or rectify the 
channel errors, conventionally, a channel coding has been carried out with respect to the encoded image signal by 
employing various error-control coding methods. In a conventional error-control coding method, extra bits are added to 

15 each predetermined set of source codes to detect and correct the channel errors during a decoding process. Correction 
of errors in this manner is possible only to a limited extent, however. If the amount off enrors oocuned in a predetermined 
set exceeds the capacity of an error correction scheme, those errors cannot be connected and the entire pixel data of a 
decoded t>lock containing such error(s) may be designated as an error without any possibility of correcting the wrong 
designation. The resulting image will then be flawed due to, e.g.. the absence of block-based pixel data that may be 

20 noticeably perceived by human vision. To deal with such problem, therefore, various error concealment schemes such 
as POCS(Projections onto Convex Sets) method have been proposed to conceal or estimate the pixel data of an error 
containing block, to thereby compensate for the degradation which may have occurred in the transmission process(see. 
e.g.. Huifang Sun et al., "Concealment of Damaged Block Transform Coded Images Using Projections onto Convex 
Sets". IEEE Transactions on Image Processing. 4. No. 4, pp. 470-477 (April 19550. However, such prior art en-or con- 

25 cealment methods remain unsatisfactory as they faW to fully tade into account the characteristics of a given image 
siganl. 

It is, therefore, a primary object of the present invention to provide an improved channel error correcting method 
which is capable of remedaying channel errors of an image signal based on the characteristics of the image signal. 

In accordance with one the present invention, there is provided a method, for use in an image signal decoding sys- 
30 tern, for concealing enors in an image signal transmitted in a l3lock-by-block compressed form, the decoding system 
including a decoding circuit for decoding the transmitted image signal into a decoded image signal on a block-by-block 
basis and detecting an en-oneous block which contains an error, the erroneous block having a predetermined number 
of sunounding t>locks without containing an error therein, each of the sunounding blocks having a multiplicity of pixels 
therein, said method comprising the steps of: 

35 

classifying each of the surrounding blocks into a monotonous, an edge or a texture class, the monotonous dass 
representing a block without containing an edge therein, the edge dass representing a block containing one edge 
therein, and the texture class representing a block containing more than one edge therein; 
classifying the erroneous block as the edge class, if only one pair of the surrounding blocks positioned opposite 
40 each other with resped to the en^oneous block is of the edge class and has an identical edge diredion transversing 
the erroneous block; and, otherwise, as either the monotonous or the texture class depending on the number of the 
surrounding blocks having either the monotonoi^ or the texture dass; and 

rectifying the erroneous block t>as6d on the classification thereof such that the pixels induded in the erroneous 
t>lock are of an identical value obtained by averaging the pixel values of the sunounding blocks it the erroneous 
45 block is of the monotonous class, such that the pixel values for the en^oneous block are determined based on the 
pixel values forming said identical edge direction transversing the erroneous block if the erroneous block is of the 
edge class, and such that the pixel values for the erroneous block are determined k>ased on the pixel values for the 
surrounding blocks of the texture class if the erroneous block is of the texture class. 

50 The above and other objects and features of the present invention will become apparent from the following descrip- 
tion of preferred emt)Odiments given in conjunction with the accompanying drawings, in which: 

Fig. 1 represents a block diagram of a channel error concealment apparatus in accordance with the present inven- 
tion; 

55 Fig. 2 provkies a detailed block diagram of a dass determination block of the channel error concealment apparatus 
of Fig. 1; 

Figs. 3A to 3G explain the operation of the class determination block shown in Fig. 1 ; 

Fig. 4 shows an enor containing block and its surrounding blocks for describing the operation of the monotonous 
dass concealment unit of Fig. 1 ; 
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Figs. 5A and 5B schematically illustrate the operation of the edge class concealment unit of Fig. 1 ; and 

Figs. 6A to 6F schematically depict the inventive concealment method in case the en^oneous block is of a texture 

class. 

5 Referring to Rg. 1 , there is shown a b>lock diagram of a channel error corx:ealment apparatus In accordance with a 
preferred embodiment of the present invention. As shown in fig. 1 . a transmitted image signal is fed to a decoding circuit 
102 through a transmission channel, wherein the transmitted image signal has been source-coded on a block-by-block 
basis and channel-coded by using a conventional enor-control coding scheme at a transmitting end. At the decoding 
circuit 102, the transmitted image signal is decoded into a decoded image signal on a block-by-block basis. Errors which 

10 may have occunred in the transmission channel are also detected and corrected at the decoding circuit 1 02. Such errors 
can be con-ected by using an error-connection scheme based on extra bits appended by the error-control coding 
scheme. However, should there occur a large number of errors than the capacity allowed in the error connection scheme, 
those errors cannot be corrected; and a whole block containing any of such unconected errors will be designated as an 
erroneous block and cannot be decoded. Outputs from the decoding circuit 102 are pixel data for the decoded blocks 

15 of the image signal and an error information signal indicating the erroneous block, which are applied to an erroneous 
block replacement unit 104. At the erroneous block replacement unit 104. in response to the error information signal 
designating the position of the erroneous block, a number of , e.g. 8, surrounding blocks thereof are retrieved from a 
storage tiierein (not shown), wherein the decoded image signal is stored; and a position information signal and the 
retrieved surrounding blocks are sent via a line L10 to a class determination blocks 106. a monotonous class conceal- 

20 ment unit 108. an edge dass concealment unit 110 and a texture class concealment unit 120. The erroneous block is 
rectified by employing one of the three concealment units, selected based on the class Information of the surrounding 
blocks. 

The class determination block 106, in oider to select a best rectifying metiiod suitable to the characteristics of the 
image signal, first classifies each of the surrounding blocks Into one of the three catagorles based on an edge pattern 

25 thereof: i.e.. a monotonous class which corresponds to a block having no edge herein, an edge dass which corre- 
sponds to a block having one edge therein and a texture dass which corresponds to a block having more that one edge 
therein, and then, determines the class of the enoneous block based on the class information of the surrounding blocks. 
The class information of the erroneous block is th^ supplied via a line Li 2 to a switch 1 18 and used therein as a control 
signal thereof. Details of the class determination block 1 06 is described witii reference to Fig. 2. 

30 Referring to Fig. 2, the pixel data and the position information signal for each of the surrounding blocks from tiie 
erroneous block replacement unit 104 are inputted to a variance calculation unit 202 and an edge detector 219 via a 
line L10, respectively At tiie variance calculation unit 202, a variance for each of the surrounding blocks is calculated 
to determine whettier fliere exists at least one edge in each of the sunrounding blocks. The variance 6^^^ may be derived 
as follows: 

35 
40 

wherein N is the total number of pixels located in a row and a column of a surrounding block, respectively; is a mean 
pixel value of the surrounding block; and X(i,j} is a pixel value at a pixel position (i.j) in the surrounding block. Botii tiie 
variance and tiie pixel values for each of the surrounding blocks are supplied to a first comparator 204. 

At the first comparator 204. tiie variance B^r ^ch surrounding block is then compared witii a first threshold 
45 value TH1 . If the variance B^a^ 's larger than THl , it is determined that the corresponding surrounding block has one or 
more edges and tiie pixel values of the surrounding block are fed to a projection block 208. If the variance is equal 
to or smaller than THl, the conesponding surrounding k^ock is considered to be of the monotonous class having no 
edges therein and a logic high signal is provided to a first counter 206. wherein the number of sunounding blocks of the 
monotonous dass is counted. 

so At the projection bUock 208, in order to determine whether each of the surrounding blocks provided from the first 
comparator 204 has one or more tiian one edges, a number of directional projections are performed with respect to 
each of the surrounding blocks applied thereto. 

Referring to Figs. 3A to 3E. there are schematically illustrated four directional projection processes carried out at 
the projection block 208 in accordance with the present invention. For an exemplary 4x4 block having 16 pixel values, 

55 the results of the four directional projections, i.e.. a vertical, a horizontal, a first diagonal and a second diagonal projec- 
tions performed as shown in Fig. 3A, are shown in Rgs. 3B to 3E, respectively. Each of projection values in Figs. 3B to 
3E is an average value of tiie pixels lying along each of projection lines which are referenced by respective numerals 
shown in fig. 3A. The projection results are then provkled to a difference calculation unit 210 and the edge detector 219. 
At the difference calculation unit 210. a first difference between tiie result of the vertical projection and that of the 
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horizontal projection and a second difference between the result of the first diagonal projection and that of the second 
diagonal projection are computed. The first difference is determined as follows: an absolute value of a difference 
between a projection value for each of the projection tines 1 to 4 of the horizontal projection and that for its correspond- 
ing projection lines referenced by an identical numeral in the vertical projection is computed first; and then, the first dif* 

5 ference is obtained by averaging the absolute values of the differences for all the projection lines. The second difference 
is determined in a similar manner. The first and the second differences are sent via a line L1 3 to a second comparator 
212 and the edge detector 219. 

At the second comparator 212, the first and the second differences are conrpared with a second predetermined 
threshold TH2. If only one of the differences is larger than or equal to TH2, it implies that there exists only one edge or 

10 a dominant edge therein and thus the surrounding block is classified as the edge class corresponding to a tAock having 
one edge therein. Output from the second comparator 212 in such a case is an edge class signal, sent to the edge 
detector 219, the edge class signal indicating that the surrounding block is of the edge class. In case both differences 
are smaller than TH2, it is interpreted as an indication of the presence of more than one non-dominant edges therein; 
and. accordingly, the block is classified as the texture class. And further, if kx)th differences are larger than or equal to 

15 TH2, the block is also classified as the texture dass. For such result implies the existence of more than one competing 
strong edges in the block. In case that the given surrounding block is determined as the texture dass, a logic high tex- 
ture class signal, indicating that the current surrounding block is of the texture class. Is sent via a line L15 to the texture 
class concealment unit 1 20 shown in Fig. 1 , and a second counter 214 wherein the number of surrounding t>locks of the 
texture class is counted. 

20 At the edge detector 219, the direction of an edge in each of the surrounding blocks of edge class is determined, 
and then the existence and the direction of an edge in the en-oneous block is determined k>ased on the edge directions 
of the surrounding blocks of edge dass as follows. First, in order to determine the edge direction for each of the sur- 
rounding bkx:ks of the edge class, tiie two differences for a given surrounding block are compared with each other. In 
response to the edge class signal from the second comparator 212. to seled a set of projections entailing a larger dif- 

25 ference, i.e.. a set of the vertical and the horizontal projections or a set of the first and the second diagonal projections; 
and then, a variance of the projection values for each projection of the selected set is calculated to determine a larger 
variance therefrom. Sut>sequently, the direction of a projection having the larger variance is selected as an edge direc- 
tion for the given surrounding block. As for tiie example shown in Figs. 3A to 3E, a set of vertical and horizontal projec- 
tions has a larger difference and the variance of the horizontal projection is greater than that of the vertical projection: 

30 and, therefore, a surrounding block having pixel values as shown in Fig. 3A is considered to have horizontal edge direc- 
tion. In case the variances of the selected set are identical incidentiy, such surrounding block is treated as of a non-edge 
class and the edge direction thereof is not determined at the edge detector 219. It should be noted that the determined 
edge of a surrounding block is assumed to pass through the center thereof. In other words, an edge in a surrounding 
t>l0Gk of edge class is quantized into one of the four edges passing through tfie center portion thereof. 

35 After determining the edge direction for each of the surrounding blocks of edge dass, such surrounding blocks are 
disposed at tiieir respective positions in response to the position information signal from the erroneous block replace- 
ment unit 104; and then an edge directions of the erroneous block is determined based on tiie edge directions of tiie 
sun^ounding blocks of edge class. Specifically, if and only if one pair of surrounding blocks positioned opposite each 
otiier witii respective to the erroneous block has edges of an identical direction, e.g., A1 and A2, B1 and B2, CI and 

40 C2. or D1 and D2 as shown in Fig. 3F, the erroneous t>lock is dassif led as the edge class and the edge direction conre- 
sponding to that pair of the surrounding blocks is assigned as tiie edge direction of the erroneous block. In such a case, 
an edge detection signal is fed to a dass determinator 218 via a line LI 7, while an edge direction signal representing 
the determined edge direction of the erroneous block is applied via a line LI 4 to tiie edge dass concealment unit 110 
shown in Fig. 1 . In all other cases, for instance, if a set of edges of the surrounding blocks such as (El . E2 and E3). (F1 . 

45 F2 and F3), (G1 . G2 and G3) and (HI , H2 and H3) shown in Fig. 3G. run parallel to the row or column direction of the 
erroneous block without passing theretiirough or if there exist more than one pair of such surrounding blocks shown in 
Fig. 3F, the erroneous block is determined as a non-edged class. 

At the class determinator 21 8, the dass of the erroneous block is determined in response to the counted numbers 
from tiie counters 206 and 214 and ttie edge detection signal from the edge detector 219. That is, if tiie edge detection 

50 signal is supplied thereto, the erroneous block is determined as tiie edge class irrespective of the counted numbers 
from the first counter 206 and 214. OthenMse, the counted numbers from the first counter 206 and the second counter 
21 4 are compared with each other and a class corresponding to the larger counted number is chosen to define the dass 
of the enroneous block, i.e., if the counted number form the first counter 206 is larger tiian or equal to that from the sec- 
ond counter 214. the erroneous block is classified as tiie monotonous class and, otiieravise, the block is classified as 

55 the texture dass. A class signal r^resenting the class of the erroneous block is sent via the line LI 2 to tiie switch 118 
as the switching control signal. 

Referring back to Fig. 1. at the monotonous class concealment unit 108, pixel values of the erroneous block are 
estimated based on tiie pixel data of all the surrounding blocks. Specifically, as for the example shown in Fig. 4. the sur- 
rounding blocks Bi to ^8 of the enroneous block DB^ are dassified into two groups, i.e., a group of edge blocks 82. B4, 
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Bs and By. and a group of corner blocks B^ , B3. B6 and Bq, the edge blocks being the blocks surrounding the erroneous 
block by an edge and the corner blocks being the blocks contiguous thereto by a corner. A representative pixel value of 
the erroneous block DB may be obtained as follows: 

^ E " w^(m^^m-^^m^^in^) w^dn^-^-m^-^-tn^^m,) Eq. (2) 

wherein E is the representative pixel value for the erroneous block DB. m. is an averaged value of the sun-ounding block 
B, with k being 1 to 8, and w and w are weight factors. The sum of two weight factors is 1/4 and w is smaller than w 

10 because the surrounding edge blocks are more influential to the erroneous block than the corner blocks. The represent- 
ative pixel value Is set as the estimated value for all the pixels of the en-oneous block and fed to the switch 118 as a first 
rectified block signal via a line L16. 

In the meanwhile, an erroneous block of the edge class is rectified by the edge class concealment unit 1 10 through 
the use of an iterative restoration algorithm described in Huifang Sun et al., supra . The iterative restoration algorithm 

IS has three steps. First, if the edge direction signal is provided from the edge detector 219 shown in Fig. 2, all the pixel 
values of the erroneous block are set to zero and a processing block consisting of the en^oneous block and Its surround- 
ing blocks is transformed by using, e.g.. a Fourier Transform, to produce a set of transform coefficients, wherein the 
edge direction of the processing t>lock in the spatial domain is reflected in a porting of the transform coeffidents of the 
set in the frequency domain along the direction normal to the edge direction. Subsequently, in order to utilize spatially 

20 correlated edge information in the processing block, transform coefficients lying outside the portion of the transform 
coefficients In the set are masked with zero values; arxf a transform block is inverse-transformed by using, e.g., an 
inverse Fourier Transform to thereby reconstruct the processing UocK wherein the transform block consists of the 
unmasked transform coefficients of the portion and the zero-valued masked transform coefficients outside the portion. 
Finally, pixel values of the reconstructed processing block corresponding to the surrounding blocks are replaced by the 

25 original pixels values thereof, thereby providing a masked processing blocK wherein the masked processing block con- 
sists of the original pixel values of the processing block of the surrounding block positions and reconstructed pixel val- 
ues at the erroneous block position. For example, if the edge of the processing block is of a diagonal direction as shown 
in Fig. 5A, pixel values of the en-oneous block are set to zeros and the entire processing block is transformed by using 
the Fourier Transform technique to thereby provide a set of transform coefficients having a DC value at (0.0) position 

30 and high frequency(Hf) components as shown in Fig. 5B. Thereafter, using the hatched portion of the transform coeffi- 
cients including the DC value along the direction orthogonal to the edge direction and zero masked transform coeffi- 
cients positioned outside the hatched portion, inverse Fourier Transform is carried out to reconstruct the processing 
block. These steps are iterated N times with respected to the masked processing block, N being a predetermined 
number, to thereby update the pixel values of the erroneous block N times to produce estimated pixel values for the 

35 erroneous block of the edge dass. The estimated pixel values are fed to the switch 1 18 as a second rectified block sig- 
nal via a line L18. 

At the texture dass concealment unit 120 having a DCT(discrete cosine transform) block 1 12, an average coeffi- 
cient calculator 1 1 4 and an IDCT(inverse discrete cosine transfbrm)block 116. the erroneous block of the texture dass 
is rectified using its surrounding block of the same type. For example, the surrounding blocks of an erroneous block DB2 

40 include monotonous blocks M-i and M2. edge blocks and E2 and texture blocks T^ to T4 as shown in Fig. 6A, the erro- 
neous block DB2 is rectified using four texture class blocks. T^ to T4. Specifically, at the DCT block 1 1 2, each of the tex- 
ture class surrounding blocks Ti to T4. is converted into a set of DCT transform coefficients in response to the texture 
class signal fed from the second comparator 212 shown in Fig. 2 within the class determination block 106, wherein the 
set of DCT transform coefficients for each of the t)locks T^ to T4 includes a DC and a predetermined number of AC coef- 

45 f icients as shown in Figs. 6B to BE. respectively, each of the coeffidents representing the magnitude of its con^espond- 
ing frequency component. 

At the average coefficient calculator 1 14, the transform coefficient of each of frequency component for the errone- 
ous block is calculated by averaging the coeffidents of the corresponding frequency component for the surrounding 
blocks of the texture class. With reference to Rg. 6F, there are illustrated transform coeffidents for the erroneous block 

50 DB2. wherein DC^ is an average transform coefficients dc^ to dC4 for the sun-ounding blocks T^ to T4 shown in Figs. 6B 
to 6E; AC1 is an average transform coeff ident of the AC coefficients acl . ac4. ac7 and acl 0; AC2, an average transform 
coefficient of the AC coeffidents ac2, ac5. ac8 and acl 1 ; and AC3. an average transform coefficient of ac3, ac6, ac9 
and acl 2, and so on. At the IDCT block 1 1 6. a set of average transform coeffidents shown in Fig. 6F is converted into 
estimated pixel values for the erroneous block of the texture dass. The estimated pixel values are fed to the switch 118 

55 as a third rectified block signal via a line L20. 

The switch 1 18, in response to the class signal from the class determination block 106. selects one of the rectified 
block signals provided from the three concealment units 1 08, 1 1 0 and 1 20. The selected rectified block signal is sent to 
the erroneous block replacement unit 104. wherein the erroneous block is replaced by the selected rectified block sig- 
nal. After processing all the erroneous blocks in the decoded image signal, the rectified image signal is provided to. e.g. . 
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a display unit(not shown) for the display thereof. 

While the present invention has been shown and described in cx>nnection with the preferred embodiments only, it 
will be apparent to those of ordinary skill in the art that many changes and modifications may be made without departing 
from the scope of the invention as defined in the appended claims. 

5 

Claims 

1. A method, for use in an image signal decoding system, for concealing errors in an image signal transmitted in a 
block-by-block compressed form the decoding system including a decoding circuit for decoding the transmitted 

10 Image signal into a decoded image signal on a block-by-block basis and detecting an erroneous fc)lock which con- 
tains an error, the erroneous block having a predetermined number of surrounding blocks without containing an 
error therein, each of the surrounding blocks having a multiplicity of pixels therein, said method comprising the 
steps of: 

IS classifying each of the sunrounding blocks into a monotonous, an edge or a texture class, the monotonous 

class representing a block without containing an edge therein, the edge class representing a block containing 
one edge therein, and the texture dass representing a block containing more than one edge therein; 
classifying the erroneous block as the edge class, if only one pair of the surrounding blocks positioned opposite 
each other with respect to the erroneous block is of the edge class and has an identical edge direction trans- 

20 versing the erroneous block; and, otherwise, as either the monotonous or the texture dass depending on the 

number of the surrourxiing blocks having either the monotonous or the texture class; and 
rectifying the erroneous block based on the classification thereof such that the pixels included in the en-oneous 
block are of an identical value obtained by averaging the pixel values of the surrounding blocks if the erroneous 
block is of the monotonous class, such that the pixel values for the erroneous block are determined based on 

25 the pixel values forming said identical edge direction transversing the erroneous block if the erroneous block is 

of the edge class, and such that the pixel values for the erroneous blocks are determined based on the pixel 
values for the surrounding blocks of the texture class if the erroneous block is of the texture class. 

2. The method of claim 1 , wherein the surrounding blocks indude four edge t>locks surrounding the erroneous block 
30 by an edge distance and four corner blocks contiguous to tiie erroneous block by a corner distance. 

3. The method of claim 2, wherein said identical value used in rectifying the erroneous block of the monotonous dass 
is obtained by multiplying a first weight factor to tiie pixel values of tiie four edge blocks and a second weight factor 
to the pixel values of the four corner t}locks and summing up the weighted pixel values. 

35 

4. The method of claim 3. wherein said enx>rs are channel errors occurred during a transmission process of tiie image 
signal. 
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FIG. 3B 
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FIG, 3D 
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FIG. 3F 
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FIG. 4 
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FIG. 6 A FIG. 6B 





T2 




Tl — 


dc-j 


ac-j 


• • • • 


T3 


DB2 


M2 






ac3 


• • • • 


El 


T4 


^2 




• 
• 


• 





FIG.6C FIG.6D 



T2— 


dc2 


ac4 


T-7 — 

• • • • ^ 


dc3 




• • • • 


ac5 






acg 


acg 


• • ♦ • 


• 
• 


• 
• 




• 


* 
• 
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